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INSTITUTIONALIZING  THE  RESULTS 


OF  THE 

INDEPENDENT  COST  ANALYSIS 


SUMMARY 

Objective  Three  of  the  Comptroller  of  the  Air  Force 
Objective  Plan  and  Strategies  is  to: 

^Assume  a  full  range  of  quality  cost,  economic  and  financial 
analysis  and  management  services  is  provided  in  support  of  the 
Air  Force  mission  .''“v 

Strategy  3. a  states; 

^Apply  analysis  capability  and  usefulness  in  the  decision 
making  process  at  all  levels.*^ 

Potentially  the  most  useful  tool  for  impacting  the  analysis 
process  in  the  area  of  cost  is  the  Independent  Cost  Analysis  (ICA). 
An  independent  test  for  reasonableness  of  a  program  office  estimate 
the  ICA  is  an  effective  tool  for  program  management. 

While  there  are  policies  in  effect  at  both  the  Department  of 
Defense  and  Air  Force  level  for  treatment  of  ICA  results,  these 
policies  do  not  lead  to  a  consistant  and  thorough  treatment  of  ICA 
results  in  program  management  and  budgeting  activities. 

This  paper  suggests  that  a  more  standardized  and  regulated 
treatment  of  1UA  results  within  the  Air  Staff  would  be  of  benefit 
to  program  financial  management,  and  serve  as  an  incentive  to  those 

who  prepare  the  estimate.  ^ 

\"  . ‘ 

INTRODUCTION 

Definition.  The  Independent  Cost  Analysis  (ICA)  is  an  Important 
step  in  the  development  and  acquisition  of  weapon  systems  by  the 
Department  of  Defense.  As  defined  by  Air  Force  Regulation  173-11. 
Independent  Cost  Analysis  Program: 

"The  ICA  is  an  independent  test  of  the  reasonableness  of  an 
official  Program  Office  (PO)  cost  estimate  of  a  major  weapon 
system  .  .  .  will  Include  as  its  major  feature  and  independent 
estimate  of  the  cost  of  the  weapon  system,  and  a  detailed 
comparison  of  the  independent  cost  estimate  with  the  official 
program  cost  estimate  including  explanations  of  significant 
differences." 
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The  services  perform  ICAs  on  all  of  their  major  system 
procurements  at  each  of  the  major  milestone  decisions,  known  as 
Systems  Acquisition  Review  Councils  (SARC),  which  occur  either  at 
service  or  DOD  level.  The  ICA  gives  decision-makers  a  current 
and  objective  evaluation  of  program  cost.  It  aids  in 
judging  what  the  ultimate  costs  will  be  and  if  the  Program 
Office  estimates,  which  normally  support  the  approved  program 
budget,  are  reasonable.  The  ICA  is  an  outside  opinion;  a 
professional,  independent  statement  of  weapon  system  expected 
cost. 

Content.  ICAs  are  tailored  for  the  particular  milestone  or 
other  event  that  led  to  their  preparation.  They  may  be  estimates 
of  development  costs,  procurement  costs,  or  operating  and 
support  costs.  The  total  program  for  all  fiscal  years  is  analyzed 
in  such  detail  that  all  system  components,  from  engines  to 
support  equipment,  are  individually  priced.  The  work  is  done  by  a 
team  of  from  half  a  dozen  to  twenty  or  more  professionals,  over  a 
period  of  from  several  months  to  half  a  year  or  more.  It  normally 
represents  the  investment  of  from  three  thousand  to  ten  thousand 
manhours,  thousands  of  dollars  in  administrative  costs,  and  the 
time  and  energies  of  many  key  service  senior  decislon«-makers.  Done 
properly,  it  is  perhaps  the  most  valid,  objective,  and  informative 
program  management  information  that  the  Air  Staff  will  ever  receive 
on  a  program,  especially  in  the  area  of  cost. 

DISCUSSION 

In  the  Air  Force,  ICAs  are  primarily  required  to  support  Either  a 
System  Acquisition  Review  Council  or  a  Defense  Resources  Board. 

ICAs  are  necessary  when  a  weapon  system  approaches  a  major  decision 
point,  for  either  a  Defense  System  Acquisition  Review  Council 
(DSARC) ,  or  an  Air  Force  Systems  Acquisition  Review  Council 
(AFSARC) .  For  DSARC  decisions  in  response  to  requests  from 
OSD  Program  Analysis  and  Evaluation  (PA&E),  the  Air  Force 
Comptroller  directs  the  appropriate  major  command  to  conduct  the 
ICA.  PA&E  is  ultimately  responsible,  through  its  role  in  the  OSD 
Cost  Analysis  Improvement  Group  (CAIG)  for  validation  of  weapon 
system  cost  analysis  for  the  Secretary  of  Defense.  This  ICA  input 
to  OSD  may  be  a  major  consideration  in  deciding  the  future  of  a 
system  at  the  DSARC. 

Validation  ICA  input  is  often  a  major  consideration  in  deciding  the 
future  of  a  system  at  the  DSARC. 

ICAs  are  also  used  to  support  the  Defense  Resources  Board  (DRB)  in 
updating  the  budget  proposals  for  a  weapon  system.  Even  though  no 
major  decision  is  pending  on  the  future  direction  for  a  program, 
the  service  or  OSD  may  require  an  ICA  to  serve  as  an  input  for 
budget  deliberations  during  a  Program  Objective  Memorandum  (POM) 
cycle.  Major  changes  in  program  content,  schedule  perturbation,  or 
cost  problems  are  some  of  the  things  that  may  cause  an  ICA.  An 
appropriate  ICA  provides  the  POM  process  and  has  the  benefit  of  the 
latest  and  often  most  comprehensive  information  for  estimates  of 
future  costs  of  programs. 


Preparation.  After  receiving  direction  from  the  Air  Force 
Comptroller,  the  procuring  activity  establishes  the  ICA  team. 

The  team  chief,  who  will  lead  the  effort  and  brief  the  results, 
develops  a  plan,  sets  in  motion  the  administrative  process,  and 
guides  the  ICA  team  in  its  work.  When  the  thousands  of  manhours 
have  produced  their  results,  he  pulls  together  the  cost  information 
and  prepares  a  briefing  to  tell  the  story.  The  first  person  to 
hear  the  story  is  the  program  manager  and  his  own  functional 
supervisors.  The  briefing  to  the  program  manager  is  a  key  point 
for  the  ICA  and  critical  to  what  will  go  forward  as  the  ICA 
recommendation.  The  ICA  and  the  program  manager's  own  cost 
estimate  are  very  likely  to  be  different,  both  in  their  sum  and 
in  their  parts.  The  ICA  team  chief  must  determine  the  reasons 
for  the  differences  and  explain  them  in  his  briefing.  It  is  one 
of  his  most  important  jobs.  When  the  differences  have  been 
examined,  and  all  adjustments  made  for  errors  of  fact  or 
calculation,  the  ICA  may  reach  only  one  of  two  conclusions. 

First,  that  the  difference  between  the  ICA  estimate  and  the 
program  office  estimate  are  relatively  minor  and  therefore  the 
ICA  team  supports  the  program  office  estimate  as  reasonable. 

Second,  the  ICA  team  concludes  the  differences  between  the 
estimates  are  significant  and  recommends  that  the  program  budgets 
be  adjusted  to  reflect  the  ICA  findings.  If  the  second 
conclusion  is  reached  the  program  manager,  in  turn,  also  has  a 
decision  to  make.  He  can  accept  the  ICA  position;  in  effect 
acknowledging  that  his  estimate  is  Incorrect,  or  he  may  stand  by 
his  estimate  and  challenge  the  ICA  results. 

Review.  The  results  of  the  ICA  are  then  briefed  to  the  procuring 
division  commander,  that  is  the  program  manager's  boss,  and  cleared 
for  presentation  to  the  service  major  command  (MAJCOM).  If  all 
differences  betweenthe  program  management  office  and  the  ICA  team 
have  not  been  reconciled,  a  decision  is  required.  The  MAJCOM 
commander  must  recommend  a  position  to  the  Headquarters  USAF 
Comptroller. 

He  must  decide  to  support  either  the  ICA  or  the  Program  Office 
and  endorse  one  estimate  as  the  MAJCOM.  The  MAJCOM  ICA  goes  to  the 
Air  Staff  where  briefings  to  the  Air  Force  CAIG,  the  Comptroller, 
Assistant  Secretary  for  Financial  Management  are  required. 

At  each  step  the  reponsible  individual  receiving  the  briefing  must 
either  endorse  the  conclusion  of  the  ICA  as  carried  forward  from 
MAJCOM,  or  make  changes  in  program  or  cost  areas  which  are  carried 
back  to  the  MAJCOM,  usually  informally,  for  coordination.  Finally, 
thre  coorindated  Air  Force  ICA  is  given  to  the  DOD  CAIG  for 
validation  and  presentation  to  the  Secretary  of  Defense. 


Post  Review.  It  should  be  noted  that  once  the  briefing  cycle  is 


completed,  the  program  cost  estimate  developed  by  the  ICA  team 
plays  no  further  specific  role  in  the  bureaucratic  process  of 
budget  or  program  management.  Yes,  the  OSD  CAIG  writes  a  report  to 
their  supervisors  on  the  ICA  results.  And  yes,  this  input  may  play 
a  part  in  any  Secretary  of  Defense  decision.  But  at  the  DSARC 
briefing  itself,  which  normally  fellows  the  presentation  of  the 


ICA,  the  ICA  may  never  be  more  than  mentioned.  The  DSARC 
principals  never  see  the  ICA  briefing.  They  may  see  a  summary 
chart  or  two,  but  the  program  manager  will  present  his  own  cost 
estimate  to  them,  and  the  ICA  may  play  a  minor  or  even  no  role  at 
all  in  the  process.  Let  me  emphasize  this  is  not  always  the  case, 
particularly  when  both  the  ICA  team  and  the  DOD  CAIG  feel  that  the 
program  manager's  estimate  is  incorrect,  nor  possibly  even  often 
the  case.  But  the  fact  remains  the  the  administrative  process  of 
the  DSARC  does  not  mandate  the  use  or  consideration  of  the  ICA  in 
assessing  the  budget  basis  for  the  program.  It's  influence  is 
limited  to  whatever  credence  the  Secretary  of  Defense  places  in  a 
summary  recommendation  on  costs  provided  him  by  the  PA&E  office. 

Shifting  from  OSD  to  the  Air  Force  the  same  general  conclusion 
must  be  reached.  Once  the  briefing  has  been  given,  the  ICA  plays 
no  further  definitive  role  in  the  weapon  system  budget  process. 

By  definitive  role  I  mean  the  Air  Staff  response  to  the  ICA 
conclusions  and  recommendations  is  not  mandated  or 
Institutionalized  beyond  the  simple  reception  of  the  briefing. 

When  an  ICA  is  received  and  its  conclusions  show  existing  budgets 
or  program  plans  do  not  appear  reasonable,  the  responses  of  Air 
Staff  agencies  are  on  a  completely  ad  hoe  basis.  The  Air  Force 
regulation  governing  the  ICA  process,  AFR  173-11,  relates  the 
responsibilities  of  all  Air  Staff  deputates  and  directorates. 

Not  one  of  these  responsibilities  extends  beyond  the  actual 
presentation  of  the  ICA*  Under  paragraph  8,  "Reviewing  the  ICA" 
the  final  act  of  the  Air  Force  CAIG  is  "prepare  a  written 
evaluation  of  the  ICA  to  be  distributed  to  the  Air  Staff  and 
Secretary  of  the  Air  Force  offices."  What  happens  next?  What  is 
done  with  the  report?  Are  the  ICA  results  to  be  implemented  or 
applied  in  some  standardized  or  consistent  manner?  Are  budgets  or 
other  planning  to  be  impacted? 

Perceptions.  The  members  of  the  ICA  team,  the  people  who  built 
and  briefed  the  estimate,  have  seen  each  reviewing  level  either 
nod  in  approval  or  order  changes  in  the  briefing  for  the  next 
level  reviewer.  When  the  ICA  comes  to  that  last  briefing,  there 
is  once  again  acceptance,  or  a  recommendation  that  changes  be 
made  and  the  briefing  be  reaccomplished .  Each  briefing  seems 
only  to  be  a  precursor  to  the  next  with  none,  even  the  last, 
representing  the  attainment  of  an  effective  outcome  other  than 
endorsement  of  the  ICA  staff.  The  briefing  team  has  expectations 
of  seeing  their  ICA  conclusions  and  recommendations  being  acted 
upon.  All  too  often  the  perception  is  that  the  whole  subject 
just  drops  into  a  hole  and  disappears.  They  would  like  to 
believe  that  now  the  Air  Staff  will  get  to  work  implementing  and 
incorporating  their  work  into  the  program  and  budget  planning. 

To  foster  this  perception,  to  provide  effective  feedback  to  the 
individuals  who  contribute  to  the  ICA  process  in  the  field  would, 
in  itself,  be  a  worthwhile  undertaking. 
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Air  Staff  and  the  ICA.  The  foregoing  paragraph  may  overstate  the 
case,  but  is  nonetheless  a  common  perception  among  those  from 
field  activities.  There  is  an  emphasis  on  bringing  the  product 
in,  rather  than  on  what  is  dene  with  it  once  it  arrives.  It  is 
certainly  true  that  ICAs  briefed  at  the  Air  Staff,  either  AFSARC, 
DSARC,  or  DRB,  are  seen  by  most  of  the  key  staff  agencies  and 
players  in  program  management.  They  are  free  to  consider  the 
impact  of  the  ICA  results  in  their  programming  and  budgeting 
activity.  The  Program  Element  Monitor  (PEM)  seeing  the  ICA  may 
note  a  FY  87  shortfall,  which  is  endorsed  by  the  program  manager. 

He  hurries  back  to  his  office  in  HQ  USAF/RD  and,  working  with  ACB 
budget  officers  who  also  saw  the  briefing,  begins  to  work  a 
Program  Budget  Decision  (PBD)  to  make  necessary  budget  changes. 

The  Comptroller  project  officer  from  ACM  prepares  a  report  which- 
will  be  circulated  and  may  play  a  part  in  future  deliberations. 

These  and  other  activities,  however,  are  loosely  structured  and 
rely  on  the  above  stated  or  informal  activity,  especially  if  the 
ICA  is  non-controversial.  There  may  not  even  be  a  Comptroller 
report,  or  it  may  not  reach  all  of  the  pertinent  offices.  The 
actions,  or  non-action,  of  the  PEM  from  RD,  the  representative  from 
AL,  the  responsible  budget  officer  from  ACB,  or  the  representative 
of  the  Assistant  Secretary  for  Financial  Management  (SAF/FM),  is 
voluntary  and  unstructured.  I  suspect  that  many  who,  view  the  ICA 
take  it  as  an  informational  briefing,  to  which  they  may  or  may  not 
respond,  and  which  Imposes  no  mandated  action  upon  them. 

Short-lived  Product.  The  ICA  briefing  and  its  conclusions  have  a 
very  short  "half-life"  in  the  Air  Staff  and  OSD  scheme  of  things. 
Program  instability,  decried  by  nearly  everyone,  is  such  that  the 
ICA  estimate,  related  to  a  specific  quantity,  schedule,  «et  of 
technical  conditions  and  program  planning,  is  soon  obsolete.  At 
each  step  of  the  Planning  Programming  Budgeting  System  (PPBS) 
cycle,  change  is  the  only  constant,  and  a  program  estimate  so 
discrete  and  specific  in  its  content  can  only  survive  in  that 
instant  between  changing  program  plans.  Indeed,  the  ICA  itself 
is  an  excellent  catalyst  for  change,  as  it  challenges 
assumptions,  uncovers  errors,  or  forces  replanning  to  accommodate 
changing  cost  estimates.  It  is  often  the  case  that  more  program 
office  effort  takes  place  after  the  ICA  briefings  than  while  the 
estimate  was  being  prepared. 

Experienced  Pentagon  managers  have  stated  the  effective  life  of 
an  ICA  is  from  three  weeks  to  several  months,  depending  upon 
surrounding  circumstances.  It  is  an  immediate,  forceful 
motivator  for  change.  Like  a  rock  thrown  into  a  pond,  it. makes  a 
big  splash  causing  ripples  of  activity  all  around  it,  but  soon 
drops  from  sight  and  is  forgotten  in  the  ongoing  existence  of  the 
acquisition  program. 


Immediate  Capture  of  Benefits.  Because  of  the  short  life  an  ICA 
can  expect  to  enjoy,  making  use  of  its  findings  must  be 
accomplished  quickly,  efficiently,  and  by  an  orderly  process.  I 
believe  the  present  system  does  not  do  that  in  the  most  effective 
ways  possible.  Actions  occur  in  response  to  the  ICA,  and  rules 
do  exist  for  using  the  product  such  as  "budgeting  to  most  likely 
cost".  But  the  process  to  achieve  such  ends  is  not  clear.  A 
mandated,  definitive  process  for  treatment  of  ICA  results  would 
make  best  use  of  the  fragile  product  within  the  time  span 
available.  It  would  also  provide  positive  evidence  to  the 
subordinate  activities  that  the  time  and  money  invested  are  being 
used  in  managing  the  program. 

Air  Staff  Oriented.  My  suggestions  for  possibly  improving  the 
treatment  of  the  ICA  product  are  oriented  towards  the  Air  Staff, 
and  more  for  DRB  ICAs  then  the  AFSARC  and  DSARC  products.  The 
budget  and  programming  process  takes  place  first  at  the  service 
level,  and  here  is  where  the  ICA  can  operate  effectively.  ICAs 
rely  on  the  use  made  of  them  by  OSD  PA&E.  Normally,  their  approach 
is  to  prepare  a  short  written  report  to  their  superiors  on  the  ICA 
product.  Experience  with  OSD  in  the  ICA  reviews  gives  one  the 
impression  that  ICAs  are  equally  useful  for  the  cost  data  they 
provide,  as  well  as  the  recommendations  brought  forward  to  aid  in 
program  decisions.  This  broader  use  of  ICAs,  perfectly  valid  for 
the  PA&E  mission,  is  less  susceptible  to  strict  guidance  on  the 
handling  of  the  ICA  product. 

CONCLUSION 

ICA  is  Valuable.  The  preparation  of  an  ICA  represents  the 
Investment  of  thousands  of  manhours,  thousands  of  dollars,  and 
considerable  managerial  expertise.  In  most  cases  it  is  a  very 
professional  product,  and  possibly,  the  very  best  piece  of 
financial  management  Information  ever  produced  on  a  given 
program.  It  is  most  valid  and  germane,  as  well  as  short-lived, 
at  the  time  of  its  delivery  to  the  service's  decision  makers.  As 
such,  it  deserves  the  very  best  treatment  and  effective 
participation  in  program  management.  Equally,  the  subordinate 
personnel  responsible  for  the  product  deserve  a  positive  response 
from  the  Air  Staff  to  their  recommendations. 

More  Structured  Role.  Because  of  constant  program  change  the 
ICA  has  short  effective  life.  Changes  indicated  by  ICAs  must  be 
put  into  motion  quickly,  and  consistent  treatment  of  ICAs  is 
necessary  to  insure  all  parties  play  their  part  in  evaluating  its 
information.  The  perception  exists  that  use  of  ICA  information 
is  too  a  greater  or  lessor  degree  haphazard  and  unpredictable. 

It  will  Improve  both  the  actual  treatment  of  the  ICA  product,  as 
well  as  the  perceptions  of  ICA  participants  if  a  more  formalized 
process  existed  for  making  use  of  the  ICA  recommendations. 


RECOMMENDATION 


The  ICA  should  serve  as  an  Institutionalized  data  point  used  in  a 
consistant  manner  by  all  pertinent  Air  Staff  agencies  for  program 
financial  management.  The  conclusions  and  recommendations  of  the 
ICA,  as  endorsed  by  the  Assistant  Secretary  of  the  Air  Force  for 
Financial  Management,  should  be  prepared  in  a  standard  format  by 
a  staff  analyst  in  the  Cost  and  Management  Analysis  Directorate. 

A  key  element  of  this  report  would  be  a  recommended  fiscal  year 
budget,  supporting  the  conclusions  of  the  ICA,  as  amended  by 
management  decision.  Other  program  issues  raised  and  resolved 
during  the  ICA  cycle  would  be  included  if  appropriate.  This 
staff  report  would  be  coordinated  through  all  relevant  AC 
directorates  and  divisions.  Then,  in  the  form  of  an  AC 
recommendation  to  SAF/FM,  it  would  be  sent  out  for  coordination 
by  other  Air  Staff  DCSs.  The  intent  is  not  that  the  ICA  outcome 
be  the  sole  valid  position  on  budget  or  other  program  Issues. 

The  PEM,  for  example,  in  cases  where  the  program  office  disagrees 
with  the  ICA  findings,  might  not  coordinate  or  be  willing  to  take 
action  consistent  with  the  ICA.  But  he  must  acknowledge  the 
report,  counter  its  findings,  and  go  on  record  formally  with  his 
own  position.  What  actually  happened  to  program  budgets  would  be 
determined  by  this  give  and  take  between  DCSs.  The  interactions 
now  carried  out  informally  and  inconsistently  would  be  documented 
and  consistent.  The  report  would  provide  a  vehicle  for  dialogue 
on  Issues  such  as  initial  spares  funding,  annual  procurement 
quantities,  and  other  program  issues.  Following  staff  coordination, 
the  final  report  would  be  transmitted  to  SAF/FM  and  possibly  by 
him  to  the  Secretary  of  the  Air  Force  as  the  Air  Staff  position 
on  the  ICA.  The  requirements  for  this  staffing  action  should  be 
Incorporated  Into  pertinent  Air  Force  regulations,  such  as  AFR 
173-11.  This  is  only  a  suggested  treatment,  and  some  other  paper 
flow  might  prove  more  appropriate  or  effective. 

Adopting  this  formal  approach  to  treatment  of  ICAs  would  achieve 
two  positive  results.  The  cost  analysis  community  outside  the 
Air  Staff,  those  who  contributed  to  the  ICA  preparation,  would  see 
a  tangible  product,  a  process  of  application  in  which  the  work 
they  brought  to  the  decision-makers  is  used  to  improve  the 
procurement  process.  More  important,  necessary  program 
management  actions  resulting  from  the  ICA  will  take  place  in  an 
orderly,  predictable  and  effective  way.  The  less  structured  and 
less  controlled  practices  now  in  being  will  give  way  to  a  more 
coherent  treatment  of  a  valuable  management  tool. 
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